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Using a pre-post design, this study evaluates the association between participating in a science game 
(NDE) and science energy knowledge and science interest. A survey was used to explore teacher 
perceptions of the game on their instructional practices. Outcome measures were collected at the 
individual level and the game was played at the team level. The results of the study indicate that for 
secondary students, post-science knowledge and interest increased very slightly, and for elementary 
students, post-interest increased slightly while post-science knowledge decreased slightly. These results 
suggest that NDE does not appear to be strongly associated with improved science knowledge and interest, 
at least for the subset of students who participated in this study, although increases were statistically 
significant. Further analysis is needed to demonstrate the game’s effect on students’ knowledge and 
interest. The majority of teachers whose students participated in the game indicated changing their 
instructional practices based on students’ inquiry and feedback. Generally, students indicated the game 
was fun and expanded their understanding of science. Longitudinal tracking of elementary students’ 
science course selection and grades as they progress through school could provide alternative measures of 
NDE program impact. 


Background Drilling Rig & Museum in Galveston and the 
Ts __ Houston Museum of Natural Sciences’ WEISS 
Energy Hall and Hall of Paleontology. Prizes were 
awarded by level (elementary and secondary) for 
high scores and participation in the game. 


In its third year, Chevron expanded funding of 
the NDE game from middle- and high-school to 
include elementary-school students in the Houston 5 ee 
Independent School District (HISD). Tasks were As technological CODIpEtEncies a US. 
modified to accommodate all student academic students continue to struggle in _ science 


levels. Students played as teams against artificial achievement (Sparks, 2013; U.S. Department of 
intelligence, competing across three levels of Education, 2011). Research based on international 


difficulty Variations in difficulty were science assessments found a decline in fourth to 
incorporated into lessons that students must master tenth-grade science performance as students 
to open options within the game. Lessons and progress through school (Leinward & Pollock, 
questions led students to game-play decisions that 2007). Psychology learning theorists explored the 
required understanding of physics, chemistry, earth role of social and cultural experiences in gaining 
science, and math concepts. A _ meta-site knowledge and the development of learning 
supplemented their learning and success for through socially-mediated activities (Cole, 1978). 
subsequent game play. The game combined Vygotsky proposes “humans are active, vigorous 
strategy, construction, and game management participants in their own existence” throughout the 
requiring players to build energy companies, gain developmental piven (Cole, hs 123). Practical 
donwnant mackcet share: aud meek the weeds oF experiences in science education through active 
cities throughout the United States. The program participation in social milieu may provide a 
included field trips to the Ocean Star Offshore 


favorable environment for learning (Driver, 
Newton, & Osborne, 2000). 
The National Science Foundation  (n.d.) 


emphasizes the importance of designing and testing 
innovative approaches to learning by applying 
technology to meet the needs of all learners. 
Research postulated that computer games enhance 
the educational experience of students, partly due 
to the multifaceted process required to learn 
science (National Research Council, 2011). Jones 
(1996) noted that the interaction between the use of 
technology and student learning may be influenced 
by various factors, including students’ interest in 
science. Bulunuz and Jarret (2009) identified the 
relationship between interest and effort, concluding 
that the more effort, the more interest, and 
interested individuals are more engaged and 
engrossed by activities because of their value. 
Ogunkola (2011) supported relationships between 
attitudes, interests, study habits, and the use of 
technology among a sample of 300 high-school 
students in Barbados. The importance of 
stimulating students’ interest has also been 
documented among middle-school students, noting 
that students who are disengaged from school are 
more likely to have poor attendance and more 
likely to drop out (Balfanz, R., Herzog, L. & Mac 
Iver, D, 2006). 

Mundie (2008) maintains that “technology has 
the potential to help reinvent the education process, 
and excite and inspire young learners to embrace 
science, math and technology” (p.1). In efforts to 
add to the body of knowledge, Plass (2011) cites 
Resnik in noting, “the best educational activities 
connect not only to important concepts but also to 
students’ interests and passions” (p.1). 

Computer games are inherently interactive and 
have, increasingly, become a new and innovative 
approach to teaching and learning science as they 
capture the attention of youth. In addition, 
computer games allow students to be transported 
into another reality and engage in activities that 
simulate the real world while exploring natural 
phenomena that cannot be directly observed 
(Games Research, 2011; Mundie, 2011). 

Previous research conducted on an NDE 
secondary-level student sample found higher 
overall mean ratings on a survey that measured 
students’ interest and attitudes in science following 
participation (Holmes, 2012). Nevertheless, there 
continues to be a gap in the research concerning the 
use of games, which features contribute to student 
learning, and whether social factors influence 
student outcomes. This study will build on the 
body of knowledge by considering background 
characteristics of students who participated in the 
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outcomes and 
elementary and 


game and assessing whether 
perceptions persist in both 
secondary student populations. 


Methodology 


All HISD elementary and secondary science 
teachers were invited to recruit students to 
participate in the NDE game during fall 2012. 
Teachers were required to register and sign a letter 
of commitment. As shown in Figure 1, the number 
of students who registered and the number of 
students who played the game in 2012-2013 more 
than doubled from the first year of implementation 
in 2010-2011 possibly due to the program’s 
expansion from secondary to include elementary 
schools. Sixteen secondary schools and 26 teachers 
were represented in the 2011-2012 data 
considering 85 secondary schools were invited to 
participate. Comparatively, 22 schools and 29 
teachers were represented in the 2012-2013 data. 
This included 10 elementary and 12 secondary 
schools. 


Measures and Variables 

Students were administered a web-based, 17- 
item multiple-choice science energy assessment 
using a pre- and post-test design. HISD science 
curriculum staff and the Tietronix’s Inc. game 
developers compiled the science assessment 
questions to align to the Texas. Essential 
Knowledge and Skills (TEKS) standards and 
content covered in the game. The instrument had 
been used in previous research conducted in 201 1— 
2012 with a comparable secondary student sample 
(Holmes, 2012). Based on the 2012-2013 data, the 
Cronbach alpha coefficient was .71, indicating 
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Figure 1. Number of all students who registered and 
participated in the NDE game, 2010-2011 through 2012— 
2013. 


good internal consistency (Pavot, Diener, Colvin, 
and Sandvik, 1991). 

Another variable analyzed in the study was 
students’ science interests. Data were captured 
during pre- and post-game participation. Survey 
items were adapted from the Program for 
International Student Assessment 2009 (PISA). 
The science interest survey was used in previous 
research with secondary students (Holmes, 2012). 
The instrument had good internal consistency in 
2012-2013, with a Cronbach alpha coefficient of 
.91. Links to both instruments were accessed at the 
game’s website. Completion of the assessment and 
survey were voluntary. However, students were 
offered incentives (e.g., tablets, trips to museum, 
trophies) to participate. 

Elementary student data were analyzed 
independently from secondary student data. This 
was the first year elementary students participated 
in the game, thus, they had less exposure to the 
survey and assessment content and data collection 
methods. Assessment questions were not modified 
due to an expectation that content would be 
covered in the game. 

Students were organized in teams. It is estimated 
that teams spent approximately 100 hours playing 
the game (F. Hughes, Tietronix Software, Inc., 
personal communication, April 10, 2013). Given 
the team format of the game, the actual number of 
hours each team member spent playing the game is 
unknown. This is important because individual 
student outcomes could be related to exposure to 
science content and experiences during the game. 


Data Analysis 

Descriptive statistics, including means and 
standard deviations, were calculated based on 
student responses to the science energy assessment 
and the science interest survey items using IBM 
SPSS software. Paired sample data measuring the 
percent correct on the pre- and post-science energy 
knowledge assessments and the mean ratings on 
the science interest survey were described in this 
evaluation. Pearson correlation coefficients 
measured the association between pre- and post- 
science energy knowledge as well as pre- and post- 
interest in science (p<.05). Linear mixed-effects 
modeling examined the effect of social factors on 
students’ science interest and science energy 
knowledge after participating in the game. This 
model took into account students nested in schools. 
The analysis was conducted at the school level 
because most schools had only one _ teacher. 
Teacher-level rather than school-level analysis 
would have reduced sample sizes, further 
compromising the reliability of the results. 
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Student Sample 

Data for 462 secondary students and 78 
elementary students who completed the four 
measures (pre- and _ post-science energy 
assessments as well as pre- and _ post-science 
interest surveys) were included in this analysis. 
Students in the matched samples were enrolled at 
six elementary and seven secondary schools (five 
middle and two high schools). Table 1 presents 
demographic characteristics of the matched student 
samples. 

The majority of elementary and _ secondary 
students were female (56.4 and 53.0 percent, 
respectively) and gifted and talented (G/T) (75.6 
and 58.4 percent, respectively). A higher 
proportion of secondary students received free or 
reduced lunch compared to elementary students 
(72.9 vs. 46.2 percent). Compared to students 
district wide, both the elementary and secondary 
student samples were more likely to be female and 
G/T, and less likely to be at risk, or LEP. The 
demographic characteristics of district-level 
students were 51.1 percent male, 15.6 percent G/T, 
62 percent at risk, 29.8 percent LEP, and 79.7 
percent economically disadvantage (PEIMS, 2012-— 
2013) analysis. 


Table 1: Demographic Characteristics of Matched- 


Paired Elementary and Secondary NDE Student 
Samples, 2013 
Elem. Secon. District 
(atts) | (6'12") 
(n = 78) | (n = 462) 

Gender % % 

Male 43.6 47.0 51.1 

Female 56.4 53.0 48.9 
Free/Reduced 46.2 72.9 79.7 
Lunch 
At Risk 21.8 12.3 62.0 
Gifted & Talented 75.6 58.4 15.6 
(G/T) 
LEP 15.4 17.1 29.8 
Elementary Grade 

Fourth 38.5 - - 

Fifth 61.5 - - 
Secondary Grade 

Middle (6"-8") - 89.4 - 

High (9-12) a 10.6 - 


What was the effect of the NDE game on 


students’ acquisition of science energy 
knowledge and interest? 
Descriptive statistics, Pearson’s correlations, 


paired t-test analysis, and linear mixed-effects 
modeling were conducted to evaluate the effect of 
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Table 2: Elementary NDE Student Means, Standard Deviations, and Inter-correlations on Science Assessment and 


Interest Predictor Variables (n=78) 


Variable M SD 1 2 3 

Post Science Energy Knowledge 55.81 19.52 .663** — 036 .087 

Predictor Variables 
Pre Science Energy Knowledge (1) 56.64 19.18 -- -.031 042 
Pre Interest Rating (2) 2.43 504 -- .743** 
Post Interest Rating (3) 2.59* 647 


**Correlation is significant at the 0.01 level (2-tailed) 
*Correlation is significant at the .05 level (2-tailed) 


the NDE game on students’ science energy 
knowledge and interests. The findings are 
presented at elementary and secondary student 
levels. 


Elementary Student Sample Findings 

Table 2 presents results for 78 elementary 
students who completed the science energy 
assessments and surveys before and _ after 
participation in the game. There was a slight 
decrease in science knowledge scores from pretest 
(M = 56.64, SD = 19.18) to post-test (M = 55.81 
and SD = 19.52), t (77) = -.46, p = .646 (two-tailed) 
for elementary students. The mean decrease was 
.84, and the 95% confidence interval of the 
difference in pre- and posttest scores ranged from - 
2.75 to 4.41. The results were not statistically 
significant at p < .05. There was an increase in the 
mean interest rating from pretest (M = 2.43, SD = 
504) to posttest (M = 2.59, SD =.657), t (77) = 
3.11, p=.003 (two-tailed). The increase was 
statistically significant at p <.05. The 95% 
confidence interval of the difference ranged from 
.056 to .254. The eta squared statistic denotes the 
magnitude of the game’s effect. The eta square was 
.11, indicating a moderate effect of the program on 
elementary students’ interest in science (Cohen, 
1988, pp. 284-7). Cause and effect relationships 


cannot be determined because a comparison group 
is necessary to infer causality. (See Appendix A 
for mean ratings by survey item.). 

The relationships between pre- and post-science 
energy knowledge and pre- and _post-science 
interest were investigated using Pearson product- 
moment correlation coefficient. Preliminary 
analyses were performed to ensure no violation of 
the assumptions of normality, linearity, and 
homoscedasticity. Table 2 shows strong, positive 
correlations between the variables. Specifically, 
based on the instrument used in this report, Pearson 
r coefficients indicate as pre-science energy 
knowledge increased, _post-science _— energy 
knowledge increased (r =.663, n = 78, p <.01). 
Also, pre-interest ratings increased as post-interest 
ratings increased (r =.743, n = 78, p <.01). While 
these relationships were statistically significant, 
only a small proportion of the variance in post- 
performance could be explained by  pre- 
performance on the instruments. 


Secondary Student Sample Findings 

Table 3 depicts results for the sample of 462 
secondary students who completed the science 
energy assessments and surveys before and after 
participation in the NDE game. A paired t-test 
analysis revealed a statistically significant increase 


Table 3: Secondary NDE Student Means, Standard Deviations, and Inter-correlations on Science Assessment and 


Interest Predictor Variables (n=462) 


Variable M SD 1 2 3 
Post-Science Energy Knowledge 65.72 19.91 A55** -.023 -.038 
Predictor Variables 
Pre-Science Energy Knowledge (1) 63.28 18.04 -- 045 093 * 
Pre-Interest Rating (2) 2.22 651 - -- .669%** 
Post-Interest Rating (3) 2.28 .692 


**Correlation is significant at the 0.01 level (2-tailed) 
*Correlation is significant at the .05 level (2-tailed) 


in the percent of science energy knowledge items 
students answered correct from pretest (M = 63.28, 
SD =18.04) to posttest (M = 65.72, SD = 19.91), 
(461) = 2.64, p = .008 (two-tailed). The 95% 
confidence interval ranged from .626 to 4.26. The 
eta squared statistic (.01), indicated the NDE 
program had a small impact on students’ science 
energy knowledge. This effect is not practically 
significant. 

Table 3 also shows that there was a statistically 
significant increase in science interest of secondary 
students from pretest (M = 2.22, SD = .651) to 
posttest (M = 2.28 and SD = .692), t (461) = 2.12 p 
= .035 (two-tailed). The 95% confidence interval of 
the difference ranged from .004 to .104. The eta 
squared statistics (.01) revealed a small effect of 
the game on secondary students’ interest in 
science. The effect is not practically significant. 
(See Appendix B for mean ratings by survey item.) 
The relationships between pre- and post-science 
energy knowledge and pre- and _post-science 
interest were investigated using Pearson product- 
moment correlation coefficient. Preliminary 
analyses were performed to ensure no violation of 
the assumptions of normality, linearity, and 
homoscedasticity. Table 3 shows strong, positive 
correlations between the variables. As pre-science 
energy knowledge increased, post-science energy 
knowledge increased (r =.455, n = 462, p <.01). 
Also, as pre-interest ratings increased, post-science 
interest ratings increased (r = .669, n = 462, p < 
01). The data also revealed a. statistically 
significant positive correlation between pre-science 
energy knowledge and post-science interest ratings 
(r = .093, n = 462, p < .05). While these 
relationships were statistically significant, only a 
small percentage of the variance in post- 
performance can be explained by pre-performance. 


Linear Mixed-Effects Modeling 

A linear mixed-effects model was used to 
analyze how much the variation from school to 
school contributed to the variation in science 
energy knowledge and science interest. Preliminary 
analyses were performed to ensure no violation of 
the assumptions of normality, linearity, multicoll 
inearity, and homoscedasiticity. Pre-science energy 
knowledge, economic status, gender, at-risk status, 
and G/T status were used as covariates in the 
model. Pre-science energy knowledge was centered 
on the group means. The variables were used as 
fixed effects and random effects to adjust for the 
variation within and between groups. These 
variables were selected because they were 
considered social factors that 
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Table 4: Elementary Student Sample Linear Mixed- 
Effects Modeling, 2013 


Post-Science Post-Science 
Energy Interest 
Knowledge 
B p B p 

Schools 43.16 | .000** | 2.30 .000** 
Eco 658 934 393 .016* 
Gender 731 354 .108 ATT 
At Risk -3.11 575 .009 .966 
G/T 427 933 .024 897 
Pre-Science 429 .890 - - 
Energy 
Knowledge 
Pre-Science - - 826 .000** 
Interest 


could have an effect on student performance 
(Beckar and Luthar, 2002). 

The elementary school dataset was relatively 
small and excluded schools with less than six 
students, thus, the minimum number of students in 
each school was six and the maximum number was 
75 students. Table 4 shows the Beta coefficients 
and p-values for elementary students. Economic 


status and pre-science interest were strong 
predictors of post-science interest among 
elementary students (p < .016 and _ p<.001, 


respectively). There was a statistically significant 
difference in post-science energy knowledge and 
post-science interest between the schools (p < 
.001). 

Regarding secondary students, pre-science 
interest was a strong predictor of post-science 
interest (p <.001). There was also a statistically 
significant difference in  post-science energy 
knowledge and post-science interest between the 
schools (p <. 001) (Table 5). 


Table 5: Secondary Student Sample Linear Mixed- 
Effects Modeling, 2013 


Post-Science Post-Science 
Energy Interest 
Knowledge 
B Pp B p 
Schools 66.22 | .000** | 2.30 | .000* 
* 
Eco - 345 059 536 
2.135 
Gender - 332 -.013 | .902 
2.585 
G/T 7.587 .033 042 577 
Pre-Science 151 919 - - 
Energy 
Knowledge 
Pre-Science - - .661 | .000* 
Interest * 


What were students’ overall impressions of the 


NDE game? 


Elementary and secondary students were asked 
to provide additional comments about the NDE 
game following participation. Student responses 
were analyzed using IBM SPSS Text Analytics to 
identify themes. Data analysis was based on 
comments of 46 elementary students and 143 
secondary students. 

The most prevalent responses centered around 
students’ positive impressions of the game. 
Students replied that the game was “fun” (n = 6), 
they liked the game (n = 4), or the game was 
“excellent” or “good” (n = 2). One student wrote, 
“The game helped me concentrate harder and 
learn a lot.” While another student commented, 
“The game is the best game about science, because 
you learn but in a fun way.” Three elementary 
students mentioned that the game was 
“complicated” or “confusing”. Among the seven 
students who mentioned “glitches” in the game, six 
still had positive comments about the game. 

Several elementary students perceived long-term 
benefits of participating in the game. Some 
comments were: 

“Tt is a great experience that will help me have a 

back up in my career. It will help me understand 

and become successful.” 


Another student wrote: 
“IT loved working on the Energy Game with my 
teammates and I think that I learned a lot about 
natural resources. The game was really fun for 
me and my whole team. I hope that you keep it 
up for a long time for other people to enjoy. It 
benefited me a lot.” 


Secondary students had similar experiences as 
elementary students playing the game. A total of 
143 secondary students out of 462 (31.0 percent) 
provided additional comments. Comments included 
the game was “fun” (n = 37) or “helpful” (n = 20); 
and “awesome,” “great,” or “amazing” (n = 13). A 
student wrote: 

“It [was] very fun and educational and whoever 

created it thank you, it’s very useful. I appreciate 

having the opportunity to play this game, thank 


” 


you. 


Another student wrote: 

“Tt’s an amazing game! It’s entertaining and gives 
us a small perspective into that career. It 
prepares us for our future.” 
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Yet another student replied: 

“T like the game and I think it is fun and I am 
learning new stuff at the same time. I like this 
game a lot involving science and other things 
that I have improved in science because of the 
science energy game. It is very helpful and fun!” 


At the same time, 14 students discussed “glitches” 
in the game; some mentioned how these glitches 
affected their outcome playing the game. 


What were NDE teachers’ perceptions 
regarding the game’s impact on_ student 
performance? 


Over the past two academic years, teachers 
whose students participated in the game were asked 
their perceptions regarding how the game 
influenced their instructional practices and student 
learning. Eight out of 26 secondary teachers (30.8 
percent) provided feedback in 2011-2012 and 16 
out of 29 elementary and secondary teachers (55.2 
percent) provided feedback in 2012-2013. It 
should be noted that several teachers responded in 
both years. 

Background information on _ training and 
professional development was captured in the 
survey. A higher proportion of teachers in 2011-— 
2012 compared to 2012—2013 agreed they received 
adequate training from game administrators (100% 
vs. 80%), had adequate technology skills to assist 
students (100% vs. 93.8%), and had sufficient 
resources to meet technical requirements (88.9% 
vs. 62.6%). Teachers’ agreement regarding 
accessing technical support was consistent in both 
years (75%). More teachers indicated having 
campus administrator support during the current 
academic year (2012-2013) compared to the 
previous year (2011-2012). 

Teachers were asked whether the game changed 
their instructional practices in specific areas. 
Results for the past two years are presented in 
Table 6. The majority of teachers indicated in both 
years their instructional practices changed in all 
areas measured. However, a higher percentage of 
teachers indicated assigning more team projects, 
cooperative group tasks, and adjusting their 
instructional practices based on student inquiry and 
feedback in 2012-2013 compared to 2011-2012. 
The most disagreement among teachers in the 
current year concerned assigning more web-based 
instruction activities compared to assigning more 
team projects and assigning more cooperative 
group tasks the previous year (26.7% and 28.6%, 
respectively). 
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Table 6: Teacher Perceptions Regarding the NDE Game’s Impact on Instruction, 2011-2012 and 2012-2013 


Strongly Disagree/Strongly 
Agree/Agree Disagree 

Since participating in the game, indicate the level of agreement % 
that you changed your instructional practices in the following % 
areas: 2011-12 | 2012-13 2011-12 | 2012-13 

(n= 8) (n= 16) (n= 8) (n= 16) 
Assign more web-based instructional activities 75.0 73.2 25.0 26.7 
Assign more team projects and cooperative group tasks 71.4 75.0 28.6 25.0 
Access more websites about science concepts 87.5 87.5 12.5 12.5 
The science game exposed my students to information that 
required the use of critical thinking skills. 100.0 100.0 0.0 0.0 
Students seemed more interested in science after participating in 0.0 6.3 
the game. 100.0 93.7 
I adjusted my instructional practices based on students’ inquiry 
and feedback. 75.0 87.5 25.0 12.5 
I used additional resources to support the concepts covered in the 
game. 87.9 81.2 12.5 18.8 
Students seemed more motivated to do well in science after 
participating in the game. 100.0 93.7 0.0 6.3 


Discussion 


Chevron’s partnership and financial support of 
the NDE game has resulted in the game’s 
expansion from 16 secondary schools in 2011— 
2012 to 22 elementary and secondary schools in 
2012-2013. The game _ created learning 
environments, which allowed students to work 
together to solve problems aligned to their 
academic levels and to science instructional 
content. Students had the opportunity to actively 
engage in learning with other students across all 
academic abilities and grade levels. 

Major objectives of the game were to increase 
students’ interest in science and their knowledge of 
science energy concepts. Exploration of the game’s 
influence on social factors, such as economic 
status, provides a perspective that may be used to 
enhance the game and equally benefit all students. 

Following the first year of the game, survey 
results of secondary students reflected perceptions 
of increased interest in being more involved in 
science-related activities during and outside of 
school (Holmes, 2012). There was also evidence 
that students increased their aspirations to pursue 
science careers and energy-related careers as adults 
based on changes in students’ attitudes from pre to 
post-survey. In 2012-2013, both elementary and 
secondary students showed overall increase interest 
in science. The results were statistically significant 
for both student groups, and practically relevant for 
elementary students. The results of the study also 
indicate that for secondary students, post-science 
knowledge and interest increased very slightly, and 
for elementary students, post-interest increased 


slightly while post-science knowledge decreased 
slightly. These results suggest that NDE does not 
appear to be strongly associated with improved 
science knowledge and interest, at least for the 
subset of students who participated in this study, 
although increases were statistically significant. 

Social factors, including school, economic status, 
gender, G/T status, and LEP status were examined 
to determine their effect on student outcomes. 
Overall, interest in science and attainment of 
science energy knowledge varied by school. 
Students who were economically disadvantaged 
had lower interest compared to non-economically- 
disadvantaged students. Student interest before 
playing the game was a significant contributor to 
their interest after the game. These findings 
highlight the need for science administrators to 
collaborate closely with teachers to identify issues 
and additional supports needed at their school to 
further enhance students’ interest and knowledge, 
particularly among students who are economically 
disadvantaged. Findings related to social factors 
also point to the possibility that students, regardless 
of background, could benefit from the game, as 
there were no practical differences among males 
and females or G/T, LEP, and at-risk students. 

The game’s effect on students and teachers was 
also noted in additional comments provided by 
participants at post-survey. In general, the 
perceptions of both groups were positive relative to 
the game’s impact on learning and _ instruction. 
Students noted the game was fun and _ helped 
expand their understanding of science. In spite of 
the fact that some teachers were involved in the 
game during previous years, the majority continued 
to indicate they modified instructional practices 


after the game based on student inquiry and 
feedback. The use of the game continued to 
motivate many teachers to seek additional 
professional development to increase their 
understanding of science energy concepts. At the 
same time, students noted technical problems and 
teachers pointed out issues meeting technical 
requirements of the game, which could have 
negatively impacted their perceptions and learning 
processes. Considering all of these factors, the 
research provides preliminary support for using 
computer games to improve student learning in 
science, given adequate resources and technical 
support. 

There were several limitations to this study. 
First, given the team format, the study lacked data 
on the actual time spent by individual students 
playing the game. This could have affected 
students’ experiences in gaining knowledge and 
influenced their interest in science at posttest. 
Continued collaboration is needed with game 
developers to develop a system for tracking 
individual student performance in order to provide 
a direct link to student interest and knowledge 
outcomes. 

Another limitation was the lack of a comparison 
group. While a pre- posttest design provided an 
alternative method for comparing students, threats 
to validity could still exist relative to knowing 
whether other external influence affected the 
change in student outcomes (e.g. history), that 
students would have changed their perceptions 
without participating in the program (maturation), 
and students’ familiarity with the surveys and 
assessments could have influenced improvements, 
(testing) (Boyd, 2002). To overcome some of the 
threats, regression analysis was used to control for 
demographic differences and pre-performance that 
may have influenced post-science interest and 
science energy knowledge. Finally, the lack of data 
at the four data collection points on all students 
who played the game resulted in substantial 
attrition. (The number of students who played the 
game was substantially lower than the number of 
students who initially registered, which was 1996 
out of 3406 students.) The recruitment process 
relied on teacher commitment, which may have 
contributed to attrition of the initial group of 
participants. 

With Chevron’s support, recommendations are to 
continue game development to foster student 
interest and build on science learning through the 
use of technology among students and teachers. 
Strategies to successfully recruit teachers and 
students in all targeted schools should be 
developed to expose more students to the 
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experience, particularly those who are struggling in 
the content area and students who are economically 


disadvantaged. 
Future analysis could include longitudinal 
tracking of science interest and knowledge, 


particularly for elementary student groups who are 
more representative of district wide students 
relative to G/T and economic status. Variables, 
such as science course-taking and science course 
grades could be considered as possible indicators 
of the game’s effect as students’ progress through 
school. Identifying a comparison group of students 
who were not exposed to the game could help 
demonstrate the effect of the game over time. 
Offering incentives may help to reduce attrition 
and increase sample sizes. 

Research has not definitively linked the use of 
media to learning. What may be more important is 
the educational content of the material that students 
are exposed to that matters most (Wainwright & 
Linebarger, 2006). To that end, using media could 
be a viable way to introduce challenging concepts 
to students, stimulate personal interest in any given 
content area, and enhance teacher instructional 
practices. 


References 
Balfanz, R. Herzog, L. & Mac Iver, D. (2006). 


Preventing Student Disengagement and Keeping 
Students on the Graduation Track in High- 


Poverty Middle-Grades Schools: — Early 
Identification and Effective Interventions. 
Educational Psychologist. 42(4), 223-235. 
Retrieved from 


http://web.jhu.edu/CSOS/images/TDMG/Prevent 
ingStudentDisengagement.pdf 

Becker, B. E., & Luthar, S. S. (2002). Social- 
emotional factors affecting achievement 
outcomes among disadvantaged students: closing 
the achievement gap. Educational Psychologist. 
37, 197-214. doi: 10.1207/S 15326985EP3704_1 

Boyd, H. (2002, November). Use Comparison 
Groups to Strengthen Your Evaluation. Paper 
presented at Audio Conference at the University 
of Wisconsin-Extension, Cooperative Extension, 
Retrieved from 
http://www.uwex.edu/ces/pdande/resources/pdf/a 
genda.pdf 

Bulunuz, N. & Jarrett, O. S. (2009). Understanding 
of Earth and Space Science Concepts: Strategies 
for Concept-Building in Elementary Teacher 
Preparation. School Science and Mathematics, 
109: 276-289. doi: 10.1111/j.1949- 
8594.2009.tb18092.x 


Cohen, J. W. (1988). Statistical power analysis for 
the behavioral sciences (2¢ ed). Hillsdale, NJ: 
Lawrence Erlbaum Associates. 

Cole, M. (1978). L. S. Vygotsky: Mind in society: 
The development of higher psychological 
processes. Cambridge, MA: Harvard Univeristy 
Press. 

Driver, R., Newton, P. & Osborne, J. (2000). 
Establishing the norms- of _ scientific 
argumentation in classrooms. Science Education, 
84(3), 287-312. 

Holmes, V. (2012). New Digital Energy Game, the 
Use of Games to Influence Attitudes, Interests, 
and Student Achievement in Science. Retrieved 
from http://eric.ed.gov. 

Hughes, F. (2013). Tietronix Software, Inc. 
Personal communication. 

Leinwand, S. and Pollock, E. (2007). Lessons 
Learned from U.S. International Science 
Performance. Retrieved from American 
Institutes for Research 
http://www.air.org/files/lessons.learned.in.scienc 
el.pdf 

Mundie, C. (2008). Microsoft Research, NYU and 
Consortium of University Partners Create First 
Scientific-Based Game Research Alliance to 
Transform Learning. Retrieved from 
http://www.microsoft.com/presspass/press/2008/ 
oct08/10-O7MSRNYUPR.mspx. 

National Assessment of Educational Progress 
(NAEP) (2011). U. S. Department of Education. 
Washington, DC: National Center for Education 
Statistics. Retrieved from 
http://nces.ed.gov/nationsreportcard/science/ 

National Research Council. (2011). Learning 
Science Through Computer Games and 
Simulations. Committee on Science Learning: 
Computer Games, Simulations, and Education, 
Margaret A. Honey and Margaret L. Hilton, Eds. 
Board on Science Education, Division of 
Behavioral and Social Sciences and Education. 
Washington, DC: The National Academies 
Press. 


New Digital Energy Program: The Use of Games, 2012-2013 


National Science Foundation (n.d.). A Teacher for 


Every Learner. Retrieved from 
http://www.nsf.gov/news/overviews/computer/co 
mp_q04.jsp 


Ogunkola, B. (2011). Teachers’ and Students’ 
Perceived Difficult Topics in the Integrated 
Science Curriculum of Lower Secondary 
Schools in Barbados. World Journal of 
Education, 1(2), 17-27. 

Plass, J. (2011). A New Way of Learning, 
Retrieved from Games for Learning Institute 
website: http://research.microsoft.com/en- 
us/collaboration/focus/cs/src_gaming.aspx 

Program for International Student Assessment 
(PISA) (2009) Mathematics Assessments. U. S. 
Department of Education. Washington, DC: 
National Center for Education Statistics. 

Sparks, S. (2013, May). Most 8th Graders Fall 
Short on NAEP Science Test. Education Week. 
Texas Education Agency. (2012-2013). PEIMS 

Edit + Reports Data Review. 

Wainwright, D. K. and Linebarger, D. L. (2006). 
Ready To Learn: Literature Review. Part 1: 
Elements of Effective Educational TV. 
Philadelphia, PA: Annenberg School for 
Communication, University of Pennsylvania; 
American Institutes for Research. Retrieved from 
http://pbskids.org/read/files/BOBPARTI- 
ElementsofSuccessfulEdTV.PDF. 


For additional information contact the HISD 
Department of Research and Accountability at 713- 


556-6700 or e-mail Research @ Houstonisd.org. 


New Digital Energy Program: The Use of Games, 2012-2013 


Appendix A 
Pre- and Posttest Interest Ratings of Elementary Student Sample 
Rating Scale: (4) Very Often; (3) Regularly; (2) Sometimes; (1) Never or hardly ever 


@ Pre @ Post 


Had fun when learning science 
Talked to my teacher about science 
Participated in science competitions 

Talked to my friends about science 

Attended science study groups 


Visited websites about science 


Participated in after-school science activities 


Talked about science to parents or family 
members 
Watched news on TV that involved science 
concepts 
Read books or magazines about science 
outside of school 


Mean Rating 


10 
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Appendix B 
Pre- and Posttest Interest Ratings of Secondary Student Sample 
Rating scale: (4) Very Often; (3) Regularly; (2) Sometimes; (1) Never or hardly ever 


Had fun when learning science 
Talked to my teacher about science 
Participated in science competitions 
Talked to my friends about science 
Attended science study groups 


— Pre 


Visited websites about science 


® Post 


Participated in after-school science activities 
Talked about science to parents or family members 


Watched news on TV that involved science concepts 


Read books or magazines about science outside of 
school 


Mean Rating 


4.0 
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